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BACKGROUND OF THE INVENTION 
The present invention is related to reception and communication of 
multimedia information, and more particularly to techniques for receiving information 
during multimedia presentations and for communicating the received information to one 
10 or more access devices. 

Multimedia presentations combining audio and video information are 
□ commonly used for communicating information in various environments such as 

q commercial environments, academic environments, and others. For example, in a 

JU commercial environment, multimedia presentations may be used during meetings, during 

Co 

,p 1 5 marketing presentations, etc. In academic environments multimedia presentations may be 

iff 

^ I used for presenting information during lectures, class proj ect presentations, etc. 

5 Conventionally, multimedia presentations are generally prepared using applications, such 

M: as Microsoft PowerPoint, which store the multimedia presentation in an accessible 

format, e.g. a multimedia presentation file. The multimedia presentation file is typically 
20 stored on a device such as a laptop computer which can be carried to the presentation 
environment. The multimedia information contained in the multimedia presentation file 
(referred to as "multimedia presentation information") may then be presented by 
connectingthe laptQp,cpmputer to an output device and by "executing" or "opening" the 
multimedia file. "Executing" or "opening" the multimedia presentation generally causes 
25 information stored in the file, such as audio and video information, to be output via the 
output device. 

The increasing popularity of multimedia presentations has also given rise 
to a need for recording the information presented during multimedia presentations to 
allow persons unable to attend the live presentation to be able to view the presentation at 
30 a later time. The recording also provides an archive which allows interested parties to 
review the presentation at a later time. Conventionally, archiving of presentations is 
typically done by storing a digital version of the multimedia presentation file. For 
example, for presentations prepared using Microsoft PowerPoint, the "*.ppt" may be 
stored. A disadvantage of this technique is that anyone reviewing the presentation at a 
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later time is typically required to use the same version of the application which was used 
to create the stored multimedia presentation. This may be difficult to satisfy at all times, 
especially when the presentation was prepared a long time ago and the application used to 
prepare the multimedia presentation is no longer available. Further, presentations 
5 prepared using applications running on a particular operating system, for e.g. Microsoft 
Windows, may not be reviewable by users using a UNIX based system. Additionally, 
due to their audio and video content, multimedia presentation files are typically very 
large, may contain many attachments, and thus consume valuable system and memory 
resources. Also, multimedia presentation files fail to capture audio information such as 

1 0 comments made by the presenter during the presentation and/or comments or questions 
asked by the attendees during the live presentation. 

Another commonly used technique for recording multimedia presentations 
is to use a video camcorder. However, this technique also has several disadvantages. For 
example, a person is required to operate the video camcorder. Also, due to the location of 

15 the video camcorder with respect to the presentation, the video recording may not be of 
good resolution and clarity. Further, since the video camcorder equipment has to be 
placed right in front of the presentation to record the presentation, valuable space, which 
could have been occupied by the attendees, is now wasted by the video camcorder 
equipment. The line of sight of die video camcorder may also be inadvertently obstructed 

20 by attendees of the presentation. Also, since the information is stored on a video cassette, 
a user needs to have access to a video player to view information stored on the video 
cassette. 

In light of the above, there is a need for a system which can receive 
information during multimedia presentations without the disadvantages associated with 
25 conventional recording techniques. It is desirable that the system be able to communicate 
the received information to devices and other consumers of the information. 

SUMMARY OF THE INVENTION 
The present invention discloses techniques for receiving information 
30 during multimedia presentations and for communicating the received information to one 
or more information access devices. According to an embodiment of the present 
invention, a presentation recorder adapter is provided for communicating information 
received during a multimedia presentation to information access devices. The adapter 
may be coupled to a first source and configured to receive multimedia presentation 
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information, comprising video information and audio information, from the first source. 
The adapter may also be configured to receive information from a second source separate 
from the first source during the multimedia presentation. For example, the second source 
may be an attendee of the multimedia presentation. The adapter may be configured to 
5 transmit the multimedia presentation information received from the first source and the 
information received from the second source to one or more information access devices. 

According to an embodiment of the present invention, the adapter may be 
configured to process the multimedia presentation information received from the first 
source and the information received from the second source to generate a first 
1 0 representation of the multimedia presentation information and the information received 
from the second source. The adapter may then be configured to transmit at least a portion 
of the first representation to the device. 

According to another embodiment of the present invention, the adapter 
may be configured to receive a request from the device requesting transmission of a first 
1 5 portion of the first representation of the multimedia presentation information and the 
information received from the second source. In response to the request, the adapter may 
kj be configured to determine the first portion of the first representation requested by the 

a u device, and to transmit the first portion of the first representation to the device. 

M- According to another embodiment of the present invention, the adapter 

5 20 may be configured to select a plurality of video frames from the video information 
2 received by the adapter, to synchronize the plurality of video frames with the audio 

information included in the multimedia presentation information received from the first 
source and audio MM^^ received from the second 

source, and to store information related to the plurality of video frames. 
25 According to an embodiment of the present invention, a system is provided 

for communicating information received during a multimedia presentation to a device. 
The system may comprise a first module configured to receive multimedia presentation 
information from a first source, the multimedia presentation information comprising 
video information and audio information, a second module configured to receive 
30 information from a second source separate from the first source during the multimedia 
presentation, a processor, a memory coupled to the processor, the memory configured to 
store a plurality of code modules for execution by the processor, and a transmitter 
configured to communicate information received by the system from the first source and 
the second source to the device. 
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According to yet another embodiment, the present invention provides a 
computer program product stored on a computer readable medium for communicating 
information received during a multimedia presentation. The computer program product 
may comprise code for receiving multimedia presentation information from a first source, 
the multimedia presentation information comprising video information and audio 
information, code for receiving information from a second source separate from the first 
source during the multimedia presentation, and code for communicating the multimedia 
presentation information received from the first source and the information received from 
the second source to one or more access devices. 

The foregoing, together with other features, embodiments, and advantages 
of the present invention, will become more apparent when referring to the following 
specification, claims, and accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figs. 1A, IB, 1C, and ID depict various environments which may 
incorporate an embodiment of the present invention; 

Fig, 2 is a simplified block diagram of a computer system suitable for use 
as a presentation recorder appliance according to an embodiment of the present invention; 

Fig. 3 is a simplified flowchart depicting processing performed by a 
presentation recorder appliance for capturing/recording information during a multimedia 
presentation according to an embodiment of the present invention; 

Fig. 4 is a simplified flowchart showing processing performed by a 
presentation recorder appliance for selecting keyframes according to an embodiment of 
the present invention; 

Fig. 5 depicts an example of a SMIL file according to an embodiment of 
the present invention; 

Fig. 6 depicts an example of a RealPix file for describing video keyframe 
tracks according to an embodiment of the present invention; 

Fig. 7 depicts an example of an entry in an augmented inverted index 
according to an embodiment of the present invention; 

Fig. 8 depicts various modules for recording information during 
multimedia presentations according to an embodiment of the present invention; 

Figs. 9A and 9B depict a user interface for retrieving the recorded 
multimedia information according to an embodiment of the present invention; 



in 



Figs. 10A and 10B depict examples of configurations in which a 
presentation recorder adapter according to an embodiment of the present invention may 
be used; and 

Figs. 1 1 A, 1 IB and 1 1C depict various embodiments of a presentation 
5 recorder adapter. 

DESCRIPTION OF THE SPECIFIC EMBODIMENTS 
The present invention provides techniques for capturing information 
during multimedia presentations. The captured information may include information, 
10 such as audio and/or video information, stored in a multimedia presentation file and 
output via an output device. The term "multimedia presentation information" will be 
used to refer to information stored in a format, such as a multimedia presentation file (or 
other format), and output using one or more output devices. According to teachings of 
the present invention, the captured information may also include information from 
' u 15 sources other than the multimedia presentation file (collectively referred to as "external 
sources"). For example, the captured information may include audio information from 
hj attendees of the multimedia presentation or the audio narration of the presenter of the 

multimedia presentation. According to an embodiment of the present invention, the 
captured information, including multimedia presentation information and information 
20 from external sources, is processed and stored in a format which facilitates efficient 
storage and retrieval. 

Figs. 1 A, IB, 1C, and ID depict various environments which may 
incorporate an embodiment of the present invention. The embodiments of the present 
invention depicted in Figs. 1 A, IB, 1C, and ID are merely illustrative of the various 
25 configurations in which the present invention may be embodied and do not limit the scope 
of the invention as recited in the claims. One of ordinary skill in the art would recognize 
other variations, modifications, combinations, and alternatives. 

Fig. 1 A depicts a presentation recorder appliance (PRA) 100 coupled to a 
multimedia presentation source (MPS) 102 via communication link 108 and to a 
30 presentation output device 104 via communication link 1 10. PRA 100 may also be 

coupled to communication network 1 12, data storage 1 14, and other devices or networks. 
MPS 102 is generally a device which is capable of storing or providing access to 
multimedia presentation information stored in a particular format. For example, MPS 102 



u 
pi 



6 



o 
m 



may be a laptop or a data processing device which stores the multimedia presentation 
information in the form of a multimedia presentation file, e.g. a PowerPoint "*.ppt" file. 

MPS 102 may also be a network which stores the multimedia presentation 
information or a device which provides access to the multimedia presentation information 
5 via the network. For example, MPS 102 may be coupled to a networked system- and be 
capable of accessing the multimedia presentation information via a communication link. 
The multimedia presentation information may include audio information, video 
information, other information, and/or combinations thereof. 

Output device 104 provides a means for outputting or presenting the 
1 0 multimedia presentation information stored by or accessed by MPS 1 02. As shown in 
Fig. 1A, output device 104 may comprise a video output device 104-a and an audio 
output device 104-b. Video output device 104-a may be configured to output video and 
graphics information included in the multimedia presentation information. Examples of 
video output device 104-a include a screen, a monitor, a television, and other like devices. 
15 Audio output device 104-b may be configured to output audio information included in the 
=P multimedia presentation information. Examples of audio output device 1 04-b include one 

y or more speakers, and other like devices. It should be apparent that various 

L configurations of output device 104, having more or less components than those depicted 

H in Fig. 1 A, may be used in conjunction with the present invention. According to an 

5 20 embodiment of the present invention, output device 104 and MPS 102 may be embodied 
^ in a single device. For example, a laptop computer may be configured to store the 

multimedia presentation file and to output the multimedia presentation information when 
the file is "executed" or "opened." 

According to the teachings of the present invention, PRA 100 may be 
25 configured to capture/record information presented during a multimedia presentation. 
The information captured or recorded by PRA 100 may include the multimedia 
presentation information stored or accessed by MPS 102 and information received from 
external sources during the presentation. In the embodiment depicted in Fig. 1 A, PRA 
100 is coupled to MPS 102 via communication link 108 and coupled to presentation 
30 output device 104 via communication link 1 10. PRA 100 may receive the multimedia 
presentation information from MPS 102 via communication link 108, store a copy of the 
multimedia presentation information before forwarding the multimedia presentation 
information to output device 104 via communication link 110. Communication links 108 
and 1 10 depicted in Fig. 1 A may be hardwire links, optical links, satellite or other 
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wireless communications links, wave propagation links, or any other mechanisms for 
communication of information. 

According to the teachings of the present invention, PRA 100 may also be 
configured to capture information from external sources 106 during the multimedia 
5 presentation. As stated above, external sources 106 may include the presenter, attendees 
of the presentation, and other sources. Examples of information received from external 
sources 106 include audio information from attendees of the presentation, audio narration 
of the person presenting the multimedia presentation, video information, and the like. 

According to an embodiment of the present invention, PRA 100 processes 
10 the multimedia presentation information received from MPS 102 and information 
received from external sources 106 and stores it in a format suitable for subsequent 
storage and retrieval. The stored format thus stores a representation of the information 
received from MPS 102 and from external sources 106. The processing performed by 
PRA 100 may include digitizing the audio and video information streams contained in the 



ffl 1 5 captured information, selecting keyframes from the video stream for storage, 



synchronizing the video information with the audio information, storing the synchronized 
video and audio information in a format suitable for later retrieval, applying speech 
recognition techniques to the audio sources, applying optical character recognition 
techniques to the video information, indexing the stored information to facilitate storage 

20 and retrieval, and several other functions. Details related to the functions performed by 
PRA 100 are described below. PRA 100 may itself be comprised of several components 
and modules which perfonn the functions. 

The information received and processed by PRA 100 may be stored on 
PRA 100 itself, or may be alternatively be stored on a data storage device 1 14 coupled to 

25 PRA 100 from where it can be accessed by a user for subsequent retrieval. The recorded 
information may be stored in a plurality of formats which can be accessed by the user 
using a variety of different interfaces. For example, the captured information may be 
accessed via a telephony interface, via a network interface, via a serial interface, and via 
other interfaces. 

30 In a networked environment, the information received and processed by 

PRA 100 may be stored on one or more data storage devices coupled to communication 
network 112 from where the recorded information can be retrieved via a plurality of 
different interfaces. Communication network 1 12 may itself be comprised of many 
interconnected computer systems and communication links. While in one embodiment, 
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communication network 1 12 is the Internet, in other embodiments, communication 
network 1 12 may be any suitable computer network. 

Fig. IB depicts another embodiment of the present invention for capturing 
information during multimedia presentations. In the embodiment depicted in Fig. IB, 

5 PRA 100 receives audio information included in the multimedia presentation information 
from MPS 102 via communication link 1 18, and receives video information included in 
the multimedia presentation information via communication link 116. According to a 
specific embodiment of the present invention, the video information may be 
communicated as a National Television System Committee ( <C NTSC") or a video graphics 

10 adapter ("VGA") compliant signal. However, other types of video and audio signals may 
also be used in conjunction with the present invention. 

PRA 100 makes a copy of the audio and video information before 
forwarding the audio and video information to output device 104 via communication links 
122 and 120, respectively. It should be apparent that in alternative embodiments of the 

15 present invention, various types of communication links may be used to communicate 
multimedia presentation information from MPS 102 to PRA 100 and from PRA 100 to 
output device 104. Communication links 1 16, 1 18, 120, and 122 depicted in Fig. IB may 
be hardwire links, optical links, satellite or other wireless communications links, wave 
propagation links, or any other mechanisms for communication of information. Similar 

20 to the embodiment depicted in Fig. 1 A, PRA 100 depicted in Fig. IB may also receive 
information from external sources 106. 

Fig. 1C depicts yet another embodiment of the present invention for 
capturing information during multimedia presentations. According to the embodiment 
depicted in Fig. 1C, PRA 100 receives the multimedia presentation information from 

25 MPS 102 via communication link 124, and output device 104 receives the multimedia 
information directly from MPS 102 via communication link 126. This type of 
configuration is generally used where communication links 124 and 126 are wireless 
communication links. However, communication links 124 and 126 may also be hardwire 
links, optical links, satellite links, wave propagation links, or any other mechanisms for 

30 communication of information. Similar to the embodiments depicted in Fig, 1 A and Fig. 
IB, PRA 100 depicted in Fig. 1C may also receive information from external sources 
106. 

Fig. ID depicts yet another embodiment of the present invention for 
capturing information during multimedia presentations. In the embodiment depicted in 



Fig. ID, output device 104 receives audio and video information included in the 
multimedia presentation information directly from MPS 102 via communication links 130 
and 128, respectively. Output device 104 may then process the information and then the 
processed audio and video information to PRA 100 via communication links 134 and 132. 
5 PRA 1 00 may also be configured to receive/capture information from external sources 
106. In the configuration depicted in Fig. ID, output device 104 may convert the various 
audio and video formats output by MPS 1 02 to a suitable standard format which is then 
forwarded to PRA 100. For example, output device 104 may convert a VGA signal 
format received from MPS 102 to a standard NTSC signal format and then communicate 
10 the NTSC signal to PRA 100. Since PRA 100 receives a standardized format, the design 
of PRA 100 is simplified since it no longer has to support the plurality of formats which 

may be output by MPS 102. 
O Fig. 2 is a simplified block diagram of a computer system 200 suitable for 

N use as a PRA 100 according to an embodiment of the present invention. Computer 

Pi I 

m 1 5 system 200 may include at least one processor 204 which communicates with a number of 
f peripheral devices via bus subsystem 202. These peripheral devices may include a 

W storage subsystem 216, comprising amemory subsystem 218 and a file storage subsystem 

L 220, user interface input devices 208, user interface output devices 210, video processing 

h subsystem 214, audio processing subsystem 212, and a network interface subsystem 206. 

5 20 The input and output devices allow user interaction with computer system 200. Network 
S interface subsystem 206 provides an interface to outside networks, including an interface 

to communication network 1 12, and may be coupled via communication network 1 12 to 
corresponding interface devices in other computer systems, for example other PRAs. As 
previously stated, communication network 112 may itself be comprised of many 
25 interconnected computer systems and communication links. These communication links 
may be hardwire links, optical links, satellite or other wireless communications links, 
wave propagation links, or any other mechanisms for communication of information. 
While in one embodiment, communication network 1 12 is the Internet, in other 
embodiments, communication network 112 may be any suitable computer network. 
30 User interface input devices 208 may include a keyboard, pointing devices 

such as a mouse, trackball, touchpad, or graphics tablet, a scanner, a touchscreen 
incorporated into the display, audio input devices such as voice recognition systems, 
microphones, and other types of input devices. In general, use of the term "input device" 
is intended to include all possible types of devices and ways to input information into 

10 



computer system 200. These devices may be used to control the operation of computer 
system 200. 

User interface output devices 210 may include a display subsystem, a 
printer, a fax machine, or non-visual displays such as audio output devices. The display 

5 subsystem may be a cathode ray tube (CRT), a flat-panel device such as a liquid crystal 
display (LCD), or a projection device. The display subsystem may also provide non- 
visual display such as via audio output devices. In general, use of the term "output 
device" is intended to include all possible types of devices and ways to output 
information from computer system 200. These output devices may be used to receive 

10 status information and other feedback information from computer system 200. 

Storage subsystem 216 stores the basic programming and data constructs 
that provide the functionality of the present invention. For example, the various modules 
for processing and recording multimedia information may be stored in storage subsystem 
216. The multimedia information captured and processed by PRA 100 may also be stored 

15 in storage subsystem 216. These software modules are generally executed by processor 
204. 

Memory subsystem 21 8 may include a number of memories including a 
main random access memory (RAM) 224 for storage of instructions and data during 
program execution and a read only memory (ROM) 222 in which fixed instructions may 

20 be stored. File storage subsystem 220 provides persistent (non-volatile) storage for 

program and data files, and may include a hard disk drive, a floppy disk drive along with 
associated removable media, a Compact Digital Read Only Memory (CD-ROM) drive, an 
optical drive, or removable media cartridges. The databases and modules implementing 
the functionality of the present invention may be stored by file storage subsystem 220. 

25 Audio processing subsystem 2 12 is responsible for capturing audio 

information received by computer system 200, and processing the audio information to 
facilitate storage of the captured audio information. Video processing subsystem 214 
may be configured to capture video information, and to process and store the video 
information for subsequent retrieval. 

30 Bus subsystem 202 provides a mechanism for letting the various 

components and subsystems of computer system 200 communicate with each other as 
intended. Although bus subsystem 202 is shown schematically as a single bus, alternative 
embodiments of the bus subsystem may utilize multiple busses. 
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Computer system 200 itself can be of varying types. Due to the ever- 
changing nature of computers, the description of computer system 200 depicted in Fig. 2 
is intended only as a specific example for purposes of illustrating the preferred 
embodiment of the present invention. It should be apparent that many other 
configurations of PRA 100 are possible having more or less components than computer 

system 200 depicted in Fig. 2. 

Fig. 3 depicts a simplified flowchart 300 showing processing performed by 
PRA 100 for capturing/recording information during a multimedia presentation according 
to an embodiment of the present invention. As shown in Fig. 3, the process of capturing 
information during a multimedia presentation may be initiated when PRA 100 receives a 
signal to commence the recording (step 302). The signal may be communicated to PRA 
100 using an input device of PRA 100, or alternatively may be communicated to PRA 
100 via a remote device. In one embodiment of the present invention, PRA 100 may 
receive the signal to start the recording directly from MPS 102. For example, when a 
multimedia presentation file is "executed" or "opened" a signal may be automatically sent 
to PRA 100 from MPS 102 instructing PRA 100 to initiate the recording process. In a 
networked environment, the signal may also be received by PRA 100 via a network 
interface. 

PRA 100 may then receive the multimedia presentation information, 
including video and/or audio information, from MPS 102 (steps 304 and 308), and 
receive information from the external sources (step 306). Steps 304, 308, and 306 may be 
performed in parallel. It should be apparent that step 304 is performed only if the 
multimedia preseni^nWfcrmatiofr vide© information, and likewise step 308 is 

performed only if the multimedia presentation information includes audio information. 
The information received from external sources may include, for example, audio 
information from the presenter of the information, audio information from the attendees 
of the presentation, for example questions or comments from the audience, audio and/or 
video information from other sources present during the presentation. 

Steps 304, 308, and 306 may be performed during the duration of the 
multimedia presentation, or until PRA 100 receives a signal to stop the recording process. 
As with the signal to initiate the recording process, a signal to stop the recording process 
may be communicated to PRA 100 using an input device of PRA 100, or alternatively 
may be communicated to PRA 100 via a remote device. According to a specific 
embodiment of the present invention, PRA 100 may receive the signal to stop the 
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recording directly from MPS 102, for example, after "execution" of the multimedia 
presentation file has completed or has been halted. In a networked environment, the 
signal may also be received by PRA 100 via a network interface. According to an 
embodiment of the present invention, the step of receiving information from external 
source may be performed even after steps 304 and 308 have been stopped. This allows 
the present invention to capture discussions etc. which may occur after execution of the 
multimedia presentation file has been completed or halted. 

The information received by PRA 100 during steps 304, 308, and 306 may 
then be processed to enable storage and subsequent retrieval of the information. 
According to step 3 10, the video information received by PRA 100 is converted to a 
digital format or digitized (step 310). The audio information received by PRA 100 may 
also be digitized (step 312). Steps 310 and 312 may be performed only if the video and 
audio signals received by PRA 100 are in analog format. Steps 310 and 312 may not be 
performed if the video and audio signals received by PRA 100 are already in digital 
format. 

PRA 100 may then select keyframes from the digitized video information 
for storage and discard the other frames (step 314). The selected frames are generally 
representative of a contiguous subset of video information. According to an embodiment 
of the present invention, PRA 100 only stores information related to the keyframes and 
discards other video information. In this manner, the amount of storage resources 
required for storage of the captured video information is reduced. In alternative 
embodiments of the present invention, for example, where storage resources are available 
in abundance, PRA 100 may also be configured to store information related to the 
keyframes as well as the other video information. 

As indicated above, storing only the keyframes information reduces the 
storage resources required for storing the video information. For example, a typical video 
stream or sequence is composed of a large number of frames, typically 60 frames per 
second, and each frame may contain 640x480 pixels. Assuming each pixel is represented 
by a byte, 307,200 bytes would be required to store each 640x480 frame. A 1-hour video 
containing 216,00 frames would thus require a massive 66GB of storage space to store 
the video information in an uncompressed format. Additionally, sequentially scanning 
through such a huge amount of stored video information to locate desired content is 
inconvenient and time consuming. By selecting a limited number of keyframes that are 
representative of a contiguous subset of video information, the present invention reduces 
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the amount of required storage space. Further, the frames chosen during step 314 provide 
a convenient means for browsing the content of the captured video and indicate points in 
the video sequence where replay could be started. 

Fig. 4 depicts a simplified flowchart 400 showing processing performed by 
5 PRA 1 00 for selecting and storing keyframes according to an embodiment of the present 
invention. As previously stated, a video sequence or stream comprises one or more video 
frames. According to flowchart 400 depicted in Fig. 4, PRA 100 compares sequential 
frames in the video sequence and selects and stores only those frames ("keyframes") that 
are significantly different from the previously selected frames. The keyframe selection 
10 process is initiated when the first frame from the captured video stream is selected and 
stored as a keyframe (step 402). The first frame is then identified or tagged as the "last 
stored frame" (step 404). For example, a variable "last_stored__frame" may be initialized 
to point to the first frame. 

PRA 100 then determines if there are more frames existing in the video 
1 5 sequence (step 406). If there are no more frames, thereby indicating that all the frames in 
the captured video sequence have been processed, keyframe selection processing is 
W terminated (step 408). If it is determined in step 406 that more frames exist, PRA 100 

u then accesses the next frame in the video sequence (step 410). 

f™ PRA 1 00 then compares the frame selected in step 41 0 with the frame 

O 20 identified or tagged as the "last stored frame" (or the frame pointed to by the 

"last_stored_frame" variable) (step 412). The comparison may involve determining the 
differences between the frames. Upon comparison, if the difference between the frame 
selected in step 410 and the '1asjt.&tQJ$d frame" exceeds a user-configurable threshold, the 
frame selected in step 410 is selected and stored as a keyframe (step 414). The frame 
25 selected and stored in step 41 0 is then designated as the "last stored frame" (the 
"last_stored_frame" variable is updated to point to the frame stored in step 410). 
Processing then continues with step 412. If however, the difference between the frames 
does not exceed the threshold value, then processing continues with step 406. 

The comparison performed in step 412 is an important determiner of 
30 system performance. If the comparison results in too many frames being selected, a large 
amount of storage space will be used for storage of the video information and browsing 
the keyframes will be inefficient (almost like re-playing the entire video stream). On the 
other hand, if the comparison results in too few keyframes being selected and stored, even 
though a large storage reduction is achieved, important information in the content of the 
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video stream may be missed. The comparison threshold value should be properly 
configured to obtain optimal comparison such that an appropriate number of keyframes 
are selected as keyframes. 

According to an embodiment of the present invention, several different 
5 methods may be used for comparing frames. These include image-based comparison 
methods, text-based comparison methods, comparison methods based on external 
stimulus, comparison methods based on environmental measurements, time-based 
sampling methods, combinations of the aforementioned methods, and others. 

According to an embodiment of the present invention using an image- 
10 based comparison method, PRA 100 uses characteristics of the image pixels of the two 
frames being compared to decide whether the frames are different. According to one 
technique, PRA 100 calculates the sum of the differences between corresponding pixels 
9 in the two frames being compared. If this value exceeds a use-configurable preset 

S ! threshold value, the frames are identified to be different from one another, and the frame 

m 15 accessed in step 410 is stored as a keyframe. The differences may be raised to the Nth 
£ power before being added to an accumulator. Another embodiment may apply an image 

W processing operator to the two frames before calculating the sum of the differences. An 

L example of such an operator is an edge detector (e.g. the Sobel detector described in R.O. 

H Duda and P.E. Hart, Pattern Classification and Scene Analysis. J. Wiley, New York, 

O 20 1973, the contents of which are herein incorporated by reference in their entirety for all 
PI purposes). 

According to another embodiment of the present invention, PRA 100 may 
use optical character recognition (OCR) text-based methods to compare the frames in step 
412 of flowchart 400. Using this technique, PRA 100 can detect changes in words 

25 between two frames which might be very difficult to detect using only image-based 

comparison methods. For example, PRA 100 may detect a word "true" being changed to 
"false" in the frames being compared. Using a text-based comparison method, PRA 100 
may decide that the two frames are different if they contain more than a fixed number of 
characters that are different. According to a specific embodiment of the present 

30 invention, in order to provide tolerance to OCR errors, the threshold may require that the 
characters that are different from each other in the two frames be contiguous. 

According to another embodiment of the present invention, PRA 100 may 
be configured to select and store a frame upon receiving stimulus or signal from MPS 102 
or some other signal source. According to an embodiment of the present invention, a 
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serial connection may be provided between MPS 102 and PRA 100. This connection 
may be used to cany information from MPS 1 02 (e.g. from a keyboard or mouse coupled 
to MPS 102, from a software application running on MPS 102, etc.) to PRA 100 
indicating when a new slide or frame is displayed. For example, every time a user 
5 touches the keyboard, moves or clicks a mouse, or explicitly causes the software 

application that is displaying the multimedia presentation to advance to the next frame, 
the next frame may be saved as keyframe. According to another embodiment of the 
present invention, this method of keyframe selection may be coupled to a video buffer 
that continuously retains "N" seconds of video. When an external stimulus is received by 
1 0 PRA 1 00, more than one keyframe may be chosen from the buffer— e.g., "M" seconds 
before the stimulus and "M" seconds after the stimulus. 

According to another embodiment of the present invention, keyframe 
selection may also be done based on signals received from sensors attached to MPS 102, 
PRA 100, output device 104, or to other devices. For example, sensors such as video 
1 5 cameras mounted on output device 1 04 or MPS 1 02 may detect when the person making 
the presentation gestures towards the device. Even though the person may not actually 
touch the device, such a motion may indicate that the person is drawing attention to the 
^ material output via output device 1 04. Accordingly, PRA 1 00 may be configured to 

select and store a frame when such a gesture is made upon receiving a signal in response 
□ 20 to the gesture from the device, e.g. a video camera. According to another technique, PRA 
% 100 may receive an environmental measurement from a device such as a microphone 

mounted on MPS 102, PRA 100, output device 104, or elsewhere. When the speaker's 
volume increases, this may indicate to the speaker is facing towards the microphone 
while speaking, and video frames during this period may be selected as keyframes and 
25 stored. 

According to another embodiment of the present invention, as an adjunct 
to the above described methods, PRA 100 may be configured to select and store 
keyframes at a fixed sampling interval (e.g. every 2 minutes) during times when the other 
methods do not choose any frames. This may help provide a level of error tolerance in 
30 that if the other methods fail to detect a significant difference, the time-based sampling 
will at least capture some data. 

Several combinations of the above-described comparison methods may be 
used by PRA 100 in alternative embodiments of the present invention. According to an 
embodiment of the present invention, PRA 100 may apply all the above mentioned 
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methods in parallel and store all the frames output by the methods as keyframes. 
According to another embodiment of the present invention, PRA 100 may be configured 
to apply all the above mentioned methods in parallel and store a frame only when "M" 
out of "N" methods select the frame as a keyframe. 
5 Referring back to Fig, 3, PRA 100 may synchronize the captured video 

information (or the selected video keyframes) and the digitized audio information (step 
316). The video keyframes and audio tracks are synchronized so that when the recorded 
multimedia information is played back, the keyframes and audio tracks will be played in 
sync as they originally occurred. According to an embodiment of the present invention, 

1 0 synchronization may be accomplished by recording a timestamp when each keyframe is 
captured and stored and at the beginning of each sound track. The timestamp may encode 
the current date and time, perhaps to fractions of seconds. 

For synchronization purposes, clock initialization is generally not an issue 
when the video and audio data are captured by using the same processor as that 

15 processor's clock may be used for the timestamp, e.g. when PRA 100 uses a single 
processor to capture the information. However, if the video information and any of the 
audio tracks are captured using different processors, then special synchronization 
procedures have to be performed if the clocks of the various processors are not set to the 
same time. These special synchronization procedures may include designating one of the 

20 processors (typically, the processor that performs video capture) as the "master." At 
startup time the difference in time between the master's clock and the non-master clocks 
is recorded. The difference value is then used by the non-master systems to calculate and 
output master-adjusted timestamps. Alternatively, the non-master systems may query the 
master for its clock value before generating a timestamp. While this method may 

25 increase the communication costs, it compensates for differences in clocks that become 
more evident over time. 

Synchronizing or cross-referencing the selected video frames with the 
audio information allows a user to retrieve sections of the multimedia presentation and 
hear the audio associated with the sections. For example, when a software application is 

30 used to playback the information recorded and stored by PRA 1 00, the software 

application may monitor the timestamps associated with the video and audio information 
to output the information at appropriate times. 

Synchronization also allows a user to randomly access the stored 
information. For example, a control such as a slider control which can be dragged along a 
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time line may be provided by a software application used to retrieve the stored 
information. The software application may map the position of the slider to a discrete 
time between the beginning and end of the multimedia presentation. The keyframe 
displayed at that discrete time may be displayed. The audio track may also be advanced 
5 or rewound to the same time and output to the user. According to an embodiment of the 
present invention, the audio streams may be broken down into subsequences or "chunks" 
where each chunk starts and ends when a keyframe is captured. A user may be allowed to 
select where to begin replay of the presentation by selecting a keyframe and the audio 
corresponding to that keyframe may be output. 
10 According to an embodiment of the present invention, the synchronization 

data for information captured during a multimedia presentation, including the timestamps 

for the audio and keyframe tracks, may be stored/represented in several ways. According 

O . ... 

;n to a specific embodiment of the present invention, a Synchronized Multimedia Interface 

Language (SMIL) format file may be used to name the audio and video tracks. Fig. 5 

ffl 1 5 depicts an example of a SMIL file for a multimedia presentation that includes one audio 

£ 

?fi track and five keyframes. A RealPix file, shown in Fig. 6, may be used to describe the 

^ video keyframe track. The file contains the names of the individual JPEG images and the 

H times during the replay when they should be displayed. 

\ 8 i Referring back to Fig. 3, PRA 1 00 may then store the audio and video 

war 

3 20 information in a format which facilitates retrieval (step 320). The multimedia 

p information may be stored in a plurality of formats which may be retrieved by a user 

using a plurality of different interfaces. For example, the multimedia information may be 
storedtas^il^ a user via a network interface 

using a web browser. Other formats in which the presentation may be stored include 
25 Synchronized Multimedia Interface Language (SMIL) format, HTML+Time format, 
Flash format, Quicktime format, text format, and other formats which can be used to 
represent recorded presentations, and the like. 

Alternatively, the audio and video information may be subjected to further 
processing (step 318). For example, the video information may be processed to extract 
30 textual information and image information from the video information. According to an 
embodiment of the present invention, optical character recognition (OCR) techniques 
may be applied to extract textual and other content information from the video 
information. The extracted information may be indexed and annotated to the video 
information to facilitate full text retrieval and keyword search capabilities for the video 
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information. Indexing and cross-referencing of the textual information with the video 
information allows a user to retrieve sections of the video information based on textual 
information. 

Audio information may also be processed in a manner similar to the video 
information. For example, voice recognition techniques may be applied to the audio 
information to recognize sources of the audio information, e.g. identify the speakers of 
the audio information, and to extract textual information from the audio information. 
According to an embodiment of the present invention, transcripts maybe generated 
corresponding to the audio information. Results from the voice recognition analysis may 
be annotated and cross-indexed with the audio information. This allows retrieval of 
segments of the audio information and their related textual information. Extraction of 
text information from the audio information also facilitates full text indexing and 
keyword searching capabilities on the audio information. 

The results obtained from applying OCR techniques to the video 
keyframes and applying speech recognition techniques to the audio information may be 
indexed for full text retrieval. Such indexing allows a user to access video or audio 
information using keyword search techniques. For example, a user may provide a 
keyword to an application used for retrieving the multimedia information, and the 
application may output video and audio information containing the keyword. 

According to an embodiment of the present invention, an "inverted index" 
may be used for indexing purposes. An inverted index contains a list of words that occur 
in the various multimedia presentations captured by PRA 100, the filenames (which may 
be used for storing the information captured) in which the words occur, and the position 
within the files where the words occur. The inverted index may be augmented with 
information indicating the media type for each indexed word. This information allows a 
software application used for retrieving the information to treat the different media types 
differently. 

Fig. 7 shows an example of an entry in an augmented inverted index. The 
media type 702 may be a document, a keyframe, audio, video, and the like. If media type 
702 is a document, "position" 704 may refer to the specific byte in the document file. If 
media type 702 is keyframe, position 704 may refer to the number of the keyframe. If 
media type 702 is audio or video, position 704 may refer to a position in time in the 
stream. 



19 



The multimedia information stored by PRA 100 may be subsequently 
accessed by a user via a plurality of interfaces. The interfaces may include a phone 
interface which may be used to access audio information from the recorded multimedia 
information. The recorded information may be also be accessed via a serial interface 
using a modem. PRA 100 may also be configured to store the recorded information in a 
format which is specialized for a particular type of retrieval device or a particular type of 
software application used for retrieval. For example, if a personal data assistance (PDA) 
such as a Palm Pilot were used to retrieve the stored information, PRA 100 may be 
configured to store the information in a storage format specialized for the PDA using 
reduced-resolution versions of the selected video keyframes. 

Fig. 8 depicts various modules for recording information during 
multimedia presentations according to an embodiment of the present invention. The 
modules depicted in Fig. 8 include a video splitter module 802, an audio splitter module 
804, a video digitizer module 806, an audio digitizer module 812, an external sources 
information capture module 810, a video keyframe selector module 808, an audio-video 
synchronizer module 814, an audio-video processor module 816, an interface module 
820, and a PRA control module 818, Fig. 8 also depicts storage subsystem 216 (also 
shown in Fig. 2) which may be used to store information used by or generated by PRA 
100. The modules depicted in Fig. 8 may be implemented in hardware or software or 
combinations thereof. The software modules may be executed by one or more processors 
in PRA 100. 

According to the embodiment shown in Fig. 8, PRA 100 is coupled to 
MPS 102 via communication links 118.and 116, and to presentation output device 104 via 
communication links 120 and 122 (similar to the configuration depicted in Fig. IB). PRA 
100 receives video information from MPS 102 via communication link 1 16 and receives 
audio information from MPS 1 02 via communication link 1 1 8, Video splitter module 802 
makes a copy of (or splits) the video information received from MPS 102 before 
forwarding the received video information to presentation output device 104 via 
communication link 120 for output via a video output device. Likewise, audio splitter 
module 102 makes a copy of (or splits) the audio information received from MPS 102 
before forwarding the received audio information to presentation output device 104 via 
communication link 122 for output via an audio output device. The copied (or split) 
video and audio streams are forwarded to video digitizer module 806 and audio digitizer 
module 812 respectively for further processing. 
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External sources information capture module 8 1 0 receives 
information/signals, for example, audio information, from external sources 106. As 
previously mentioned, the information from external sources 106 may include for 
example the audio narration of the presenter, questions or comments from the 
5 presentation attendees, and audio signals from other sources. External sources 

information capture module 810 generally includes one or more listening devices such as 
microphones which capture audio information from the external sources. The captured 
audio information is forwarded to audio digitizer module 804 for further processing. 
According to alternative embodiments of the present invention, external sources 
1 0 information capture module 8 1 0 may also include a video camera to capture video 

information from external sources. The video information captured from external sources 
_ 106 is forwarded to video digitizer module 806 for further processing, 

•jj Video digitizer module 806 is responsible for converting analog video 

2] signals to a digital format. The digitized video information is then forwarded to video 

60 1 5 keyframe selector module 808 for further processing. Audio digitizer module 8 12 is 
yi responsible for converting analog audio signals to a digital format. The digitized audio 

information is then forwarded to audio-video synchronizer module 814 for further 
H processing. As previously stated, audio and/or video digitizing may not be required if the 

y audio and video information received from MPS 102 or from external sources 106 is 

!? 20 already in digital format. 

I i 

□ Video keyframe selector module 808 is responsible for processing the 

digitized video information to select keyframes from the video information and to discard 
the other frames. As mentioned above, selection of keyframes rather than all the frames 
reduces the amount of memory needed for storage of the video information. This in turn 

25 reduces the amount of resources needed to store the multimedia presentation information. 
Various techniques which may be used by the present invention to select keyframes have 
been described above. 

Audio-video synchronizer module 814 is responsible for receiving 
information related to the video keyframes selected by video keyframe selector module 

30 808 and synchronizing the keyframes with digitized audio information received from 
audio digitizer 812. As previously mentioned, synchronizing or cross-referencing the 
selected video frames with the audio information allows a user to retrieve sections of the 
multimedia presentation and hear the audio associated with the sections. 
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According to an embodiment of the present invention, the synchronized 
audio and video information may be stored by audio- video synchronizer module 814 in 
storage subsystem 216, data storage 1 14, or some other storage location. The stored 
information may then be accessed by a user in various formats using different interfaces 
5 820. 

According to another embodiment of the present invention, the audio- 
video information may then be forwarded to audio-video processor 816 for further 
processing. Audio-video processor 816 may comprise a single processor or a plurality of 
processors. The processing performed by audio-video processor 816 may include 
10 extraction of textual information from the video and audio information, cross-referencing 
or indexing the extracted textual information with the video/audio information, and other 
types of processing. Various techniques such as OCR techniques and voice recognition 
techniques, as described above, may be used to facilitate extraction of textual and other 
information from the audio and video information. Other known signal processing 
1 5 techniques may also be used by audio-video processor 81 6 to extract information from 
the audio and video information. 
m The multimedia information and the information extracted by audio-video 

processor 816 may then be stored in a plurality of formats to facilitate subsequent 
retrieval by a user. As stated above, various different formats may be used for storing the 
0 20 information including SMIL format, HTML+Time format, Flash format, Quicktime 
format, text format, and other formats which can be used to represent recorded 
presentations. The recorded multimedia information may be stored in storage subsystem 
216, or in data storage device 1 14 coupled to PRA 100. The information may also be 
stored on data storage devices (not shown in Fig. 8) coupled to PRA 100 via 
25 communication network 1 12. Audio-video processor 816 may use encoding and 
compression techniques to minimize the number of resources needed for storing the 
information. 

Interface module 820 enables a user to access the recorded information 
stored by PRA 100 via a plurality of interfaces. These interfaces may include a telephone 
30 interface 820-a, a network interface 820-b, a serial interface 820-c, and other interfaces 
820-d. Other interfaces 820-d may include interfaces which allow information to be 
retrieved via a cellphone, via a PDA, and other like communication techniques. 

PRA control module 818 is configured to provide mechanisms for 
controlling the functionality of PRA 100. For example, PRA control module 818 may 
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provide controls to instruct PRA 100 to start or stop the recording process. The start/stop 
controls may be activated using input devices coupled to PRA 100 or other devices such 
as remote control devices. PRA control module 818 may also include controls for 
playing back the recorded information using output devices coupled to PRA 100. Other 
5 controls typically associated with video and audio equipment may also be included in 
PRA control module 116. Status information related to PRA 100 may also be displayed 
or output using PRA control module 818. 

The modules depicted in Fig. 8 are merely illustrative of an embodiment of 
the present invention and do not limit the scope of the invention as recited in the claims. 
10 One of ordinary skill in the art would recognize other variations, modifications, 
combinations, and alternatives. 

Figs. 9A and 9B depict a user interface 900 for retrieving the recorded 
multimedia information according to an embodiment of the present invention. User 
interface 900 may be used in conjunction with a web browser and may connect to a web 
1 5 server process executing on PRA 100. For example, user interface 900 may be presented 
7 1 as a web page. User interface 900 may allow a user to access the stored multimedia 

f information, browse the information, search the stored information, randomly access 

u contents of the stored information based on date, time, etc., perform text-based searches, 

p. and perform other functions. 

Q 20 As described above, according to an embodiment of the present invention, 

~ information related to each individual presentation may be stored in HTML format that is 

derived from the SMEL format. According to this embodiment, the HTML may include 
thiunbnail images of the keyframes and links to other media types, and computational 
methods that enable various communication methods. For example, user interface 900 
25 depicted in Fig. 9 A displays a presentation in which six keyframes 902- 1 , 902-2, 902-3 , 
902-4, 902-5, and 902-6 corresponding to the multimedia presentation are displayed. 
User interface 900 also includes several features for accessing information corresponding 
to the presentation. For example, a user may invoke a media player application, such as a 
Windows Media Player or a RealMedia Player, by clicking or selecting the "Real Media" 
30 button/hypertext link 906. The media player invoked upon selecting button 906 may be 
used to replay the presentation as a video and play the accompanying audio. Selecting 
"OCR results" 904 button/link allows the user to access results of OCR techniques 
associated with the presentation. The user may print information related to the 
presentation by selecting "Print" button/link 908. The user may publish information 
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related to the presentation by selecting "Publish on the WWW" button/link 914. The user 
may email the presentation information by selecting "Email button/link 912. 

User interface 900 may also include a text search window 910 which 
allows the user to search the presentation information. Text typed in window 910 is 
converted to queries that may be matched against, for example, the augmented inverted 
index described above. Filenames of documents that match the queries and the 
corresponding HTML representations of the documents may be returned to the user 
together with an indication of the medium that contained the query text. For example, in 
response to the user typing the "Summary" in window 910 (as shown in Fig. 9B), a 
section of keyframe 902*2 containing the word "Summary" may be highlighted as shown 
in Fig, 9B. Search terms in an audio soundtrack may be expressed by outlining the 
closest keyframe. Alternatively, user interface 900 may display the transcript of the audio 
track in which the search terms occur with the search terms highlighted. Users may click 
on the highlighted terms in the transcript and the video associated with the presentation 
may be played from that time. Alternatively, according to an embodiment of the present 
invention, a snippet of the audio track containing the search term may be played. The 
transcript or audio snippet may be linked to a media player. Various other techniques 
may also be used to present the results of a search query to the user. 

User interface 900 depicted in Figs. 9A and 9B is merely illustrative of an 
embodiment of the present invention and does not limit the scope of the invention as 
recited in the claims. One of ordinary skill in the art would recognize other variations, 
modifications, combinations, and alternatives. 

PRESENTATION RECORDER ADAPTER 

According to the teachings of the present invention, techniques are 
disclosed for recording information during multimedia presentations and for facilitating 
communication of the recorded information to one or more information access devices. A 
Presentation Recorder ADapter (PRAD) is disclosed which may be configured to 
receive/record information, including audio information and video information, during 
multimedia presentations and to communicate the recorded information to information 
access devices. The information access devices may include devices such as PRA 100, 
computer systems, hand-held devices, personal digital assistants (PDAs), and other data 
processing devices. In general, use of the term "information access devices" is intended 
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to include all possible types of devices which can access information or to which 
information can be communicated. 

Figs. 10A and 10B depict examples of configurations in which a PRAD 
according to an embodiment of the present invention may be used. The embodiments of 
the PRAD depicted in Figs. 10A and 10B are merely illustrative of the various 
configurations in which the present invention may be embodied and do not limit the scope 
of the invention as recited in the claims. One of ordinary skill in the art would recognize 
other variations, modifications, combinations, and alternatives. 

As depicted in Fig. 10A, a PRAD 1000 may be configured to receive audio 
information included in the multimedia presentation information from MPS 102 via 
communication link 1006, and to receive video information included in the multimedia 
presentation information via communication link 1004. PRAD 1000 may make a copy of 
or buffer the audio and video information before forwarding the audio and video 
information to output device 104 via communication links 1010 and 1008, respectively. 

PRAD 1000 may also be configured to receive information from external 
sources 106 during the multimedia presentation. As stated above, external sources 106 
may include the presenter, attendees of the presentation, and other sources. Examples of 
information received from external sources 106 include audio information from attendees 
of the presentation, audio narration of the person presenting the multimedia presentation, 
video information, and the like. 

According to an embodiment of the present invention, PRAD 1000 may 
process the information received from MPS 102 and from external sources 106 and store 

devices 1002. The information processed by PRAD 1 000 may be stored on PRAD 1000 
itself, or may be alternatively be stored on a data storage device accessible to PRAD 
1000. The recorded information may be stored in a plurality of formats. 

As indicated above, according to an embodiment of the present invention, 
PRAD 1000 is configured to communicate the recorded information to access devices 
1002. The information may be communicated to information access devices 1002 in 
various modes. For example, according to a first mode, PRAD 1000 may continuously 
transmit the information to information access devices 1002 via communication links 
1012 and 1014 during the multimedia presentation. In this mode, an information access 
device remotely located from the location of the multimedia presentation may be able to 
watch and hear the multimedia presentation or portions thereof in real time. In alternative 
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modes, the information may be buffered by PRAD 1000 and communicated to 
information access devices 1002 upon receiving requests from the information access 
devices. A command API may be provided which may be used by information access 
devices 1002 to receive information buffered by PRAD 1000. 
5 Fig. 10B depicts another configuration incorporating PRAD 1000 

according to an embodiment of the present invention. In the embodiment depicted in Fig. 
1 0B, output device 1 04 receives audio and video information included in the multimedia 
presentation information directly from MPS 102 via communication links 1022 and 1020, 
respectively. Output device 1 04 may then process the received information and then 
1 0 forward the processed audio and video information to PRAD 1 000 via communication 
links 1018 and 1016, respectively. PRAD 1000 may also be configured to 
receive/capture information from external sources 106. PRAD 1000 may process the 
S recorded information and then communicate the information to information access 

%A devices 1002 via communication links 1012 and 1014. In the configuration depicted in 

15 Fig. 1 0B, output device 104 may convert the various audio and video formats output by 
=p MPS 102 to a suitable standard format which is then forwarded to PRAD 1000. For 

hj example, output device 104 may convert a VGA signal format received from MPS 102 to 

f : a standard NTSC signal format and then communicate the NTSC signal to PRAD 1 000. 

M Since PRAD 1000 receives a standardized format, the design of PRAD 1000 is simplified 

yj 

f=! 20 since it no longer has to support the plurality of formats output by MPS 1 02. 

2 Communication links 1004, 1006, 1008, 1010, 1012, 1014, 1016, 1018, 

: 5 

1 020, and 1022 depicted in Figs. 10A and 10B may be hardwire links, optical links, 
satellite or other wireless communications links, wave propagation links, or any other 
mechanisms for communication of information. In a specific embodiment, 

25 communication links 1012 and 1014 are wireless communication links. The 

communication links depicted in Figs. 10A and 10B are merely illustrative of the various 
configurations in which the present invention may be embodied and do not limit the scope 
of the invention as recited in the claims. One of ordinary skill in the art would recognize 
other variations, modifications, combinations, and alternatives. For example, single or 

30 multiple communication links may be used to communicate information between the 
various components shown in Figs. 10A and 10B. 

Fig. 1 1 A depicts various modules which may be included in PRAD 1000 
according to an embodiment of the present invention. The modules depicted in Fig. 1 1 A 
include a video splitter module 1 102, an audio splitter module 1 104, a video digitizer 
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module 1 1 08, an audio digitizer module 1 1 1 0, an external sources information capture 
module 1 106, a video keyframe selector module 1 1 12, a URL assignment module 1 1 16, a 
timestamp module 1 1 14, a PRAD control module 1 120, storage subsystem 1 1 18, and 
transceiver 1 122. The modules depicted in Fig. 1 1 A may be implemented in hardware or 
5 software or combinations thereof. The software modules may be executed by one or 
more processors of PRAD 1000. 

As shown in Fig. 1 1 A, PRAD 1000 may receive video and audio 
information included in the multimedia presentation information from MPS 102. Video 
splitter module 1 102 makes a copy of (or splits) the video information received from 
1 0 MPS 1 02 before forwarding the received video information to presentation output device 
104. Likewise, audio splitter module 1 104 makes a copy of (or splits) the audio 
information received from MPS 102 before forwarding the received audio information to 
^ presentation output device 104. 

SJ The copied (or split) video and audio streams may be forwarded to video 

m 1 5 digitizer module 1 108 and audio digitizer module 1110 and/or to transceiver 1 122. When 
PRAD 1000 operates in a mode during which the recorded information is transmitted to 

yi 

yj information access devices 1002 in a continuous manner during the multimedia 

jj^ presentation, the copied audio and video information is forwarded to transceiver 1 122. 

p t Transceiver 1 122 may then communicate the information to information access devices 

3:1 

O 20 1 002. According to an embodiment of the present invention when PRAD 1000 is not 
~ operating in the continuous transmission mode, the copied information may be forwarded 

xssS 

to video digitizer module 1 108 and audio digitizer module 1 1 10 for further processing. In 
alternative embodiments, the copied information may be forwarded to video digitizer 
module 1 108 and audio digitizer module 1110 even when PRAD 1000 is operating in the 

25 continuous transmission mode. 

External sources information capture module 1 106 receives 
information/signals, for example, audio information, from external sources 106. As 
previously mentioned, the information from external sources 106 may include audio 
narration of the presenter, questions or comments from the presentation attendees, audio 

30 signals from other sources, video information, and the like. External sources information 
capture module 1 106 may include one or more listening devices such as microphones 
which capture audio information from external sources 106. External sources information 
capture module 1 106 may also include a video camera to capture video information from 
external sources 106. 
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External sources information capture module 1 106 may forward the 
information captured from external sources 106 to video digitizer module 1 108 and audio 
digitizer module 1110, and/or to transceiver 1 122. When PRAD 1000 operates in a 
continuous transmission mode, the external sources information may be forwarded to 
5 transceiver 1 122. Transceiver 1 122 may then communicate the information to 

information access devices 1002. The information may be forwarded to video digitizer 
module 1 108 and audio digitizer module 1 1 10 for further processing when PRAD 1000 is 
not operating in the continuous transmission mode. In alternative embodiments, the 
information may be forwarded to both the video digitizer module 1 108 and audio digitizer 
1 0 module 1110 and also to transceiver 1 1 22. 

Video digitizer module 1 108 is responsible for converting analog video 
signals to a digital format. The digitized video information is then forwarded to video 
5 keyframe selector module 1 1 12 for further processing. Audio digitizer module 1 1 10 is 

Q responsible for converting analog audio signals to a digital format. The digitized audio 

m 1 5 information may then be forwarded to timestamp module 1 1 14 for further processing. As 
;P previously stated, audio/video digitizing may not be required if the audio and video 

kj information received from MPS 1 02 or from external sources 1 06 is already in digital 

U format. 

N= Video keyframe selector module 1 1 12 is responsible for processing the 

p 20 digitized video information to select keyframes from the video information and to discard 

y the other frames. As mentioned above, selection of keyframes rather than all the frames 

□ 

reduces the amount of memory needed for storage of the video information. This in turn 

Various techniques which may be used by PRAD 1000 to select keyframes have been 

25 described above with respect to PRA 1 00. 

After video keyframes have been selected by video keyframe selector 
1 1 12, each keyframe may be assigned a unique identifier which may be used to access a 
particular keyframe. According to an embodiment of the present invention, each 
keyframe may be stored as a web page and assigned a uniform resource locator (URL). A 

30 user using a browser may then access a particular keyframe by providing the URL 

information for the particular keyframe to the browser. URL assignment module 1 1 16 is 
responsible for creating web pages for the keyframes and assigning URLs to the web 
pages. 
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Timestamp module 1 1 14 time stamps the video and audio information. 
For example, each web page containing a keyframe may be time stamped. Likewise, 
audio information may be time stamped. PRAD 1000 may use the time stamps associated 
with the audio and video information to synchronize the audio and video information. 

5 Time stamping allows an information access device to access sections of the multimedia 
presentation based upon a timeline and hear the audio information associated with the 
sections. According to an embodiment of the present invention, the time stamped audio 
and video information may be stored in storage subsystem 1 1 18 or in some other storage 
location accessible to PRAD 1000. Encoding and compression techniques may be used to 

1 0 minimize the number of resources needed for storing the information. 

Transceiver 1 122 may be configured to communicate the information 
received from MPS 102 and from external sources 106 to information access devices 
1002. As indicated above, information may be transmitted to information access devices 
1 002 in various ways. In a first mode, transceiver 1 122 may be configured to transmit the 

15 information to information access devices 1002 in a continuous manner. The information 
transmitted in this continuous transmission mode may include the audio and video 
information received by transceiver 1 122 from video splitter 1 102, audio splitter 1 104, 
and from external sources information capture module 1 106. The transmitted information 
may also include information stored in storage subsystem 1118. For example, transceiver 

20 1 122 may be configured to transfer URLs and audio information associated with the 
URLs. In alternative embodiments, transceiver 1 122 may be configured to transfer only 
audio information or only video information. 

Acceding to another mode, transceiver 1 122 may be configured to 
communicate information to information access devices 1002 based upon requests 

25 received from information access devices 1002. In this mode, transceiver 1 122 may be 
configured to receive requests/commands from information access devices 1002 
requesting transmission of information stored in storage subsystem 1118. In response, 
transceiver 1 122 may transmit audio and/or video information as requested by the 
information access device. 

30 According to an embodiment of the present invention, PRAD 1000 may 

support various commands which may be used by information access devices 1002 to 
request transmission of the audio and video information stored by PRAD 1000 or portions 
thereof. For example, commands may be provided requesting PRAD 1000 to transmit 
only audio information, only video, only information received from MPS 102, only 
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information received from external sources 106, and combinations thereof. For example, 
an infonnation access device may provide one or more URLs and request transmission of 
video information corresponding to the URLs. Commands may also be provided 
requesting PRAD 1000 to transmit information based upon a timeline. For example, an 
5 information access device may provide a starting and ending time, and request 

transmission of information between the starting time and ending time. An infonnation 
access device may also request transmission of audio and/or video information for the 
first "N last "N etc. seconds of the multimedia presentation. An information access 
device may also request transmission of video information corresponding to the first "N," 
10 last "N, etc. video keyframes. Various other techniques may also be used to access the 
audio and video information stored by PRAD 1000. 

PRAD control module 1 120 is configured to provide mechanisms for 
O controlling the functionality of PRAD 1000. The controls may be activated using input 

Si devices coupled to PRAD 1 000 or other devices such as remote control devices. PRAD 

l M 15 control module 1120 may also display status information related to PRAD 1000. 
± The modules depicted in Fig. 1 1 A are merely illustrative of an 

hi embodiment of PRAD 1000 and do not limit the scope of the invention as recited in the 

L. claims. One of ordinary skill in the art would recognize other variations, modifications, 

r* combinations, and alternatives. 

f==. 20 Fig. 1 IB depicts another simplified embodiment of PRAD 1000. The 

Ej design of this embodiment is simpler than the embodiment depicted in Fig. 1 1 A as PRAD 

1000 does not contain the video keyframe selector 1112 and URL assignment 1116 
modules. M the embodkient^d^iGted^in Fig. 1 15, .the«digitized vidminf@Emation is 
communicated to timestamp module 1114. As with the embodiment depicted in Fig. 
25 1 1 A, PRAD 1 000 may transmit information to information access devices 1 002 in 
various modes. 

Fig. 1 1C depicts yet another simplified embodiment of PRAD 1000. In 
the embodiment depicted in 1 1C, the information recorded by PRAD 1000 is not stored 
or buffered by PRAD 1000. The information received by PRAD 1000 from MPS 102 and 
30 external sources 106 is communicated to transceiver 1 122 which transmits the 

infonnation to information access devices 1002. Accordingly, PRAD 1000 depicted in 
Fig. 11C transmits information to information access devices 1002 in' a continuous 
manner. 
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Although specific embodiments of the invention have been described, 
various modifications, alterations, alternative constructions, and equivalents are also 
encompassed within the scope of the invention. The described invention is not restricted 
to operation within certain specific data processing environments, but is free to operate 
within a plurality of data processing environments. Additionally, although the present 
invention has been described using a particular series of transactions and steps, it should 
be apparent to those skilled in the art that the scope of the present invention is not limited 
to the described series of transactions and steps. 

Further, while the present invention has been described using a particular 
combination of hardware and software, it should be recognized that other combinations of 
hardware and software are also within the scope of the present invention. The present 
invention may be implemented only in hardware, or only in software, or using 
combinations thereof. 

The specification and drawings are, accordingly, to be regarded in an 
illustrative rather than a restrictive sense. It will, however, be evident that additions, 
subtractions, deletions, and other modifications and changes may be made thereunto 
without departing from the broader spirit and scope of the invention as set forth in the 
claims. 
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